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• • • ^ 

Abstract 

• ^ 1 V 

Due to the recent Federal Redefinitfion of "basic skills," large scale 
d^ssessments of speaking and listening proficiencies wVll prol iferate. 
In, planning assessment procedures, educators must be sensitive pot only 
to psychometric validity, but also to pedagogical validity - the effects' 
of testing on curricular content and instrijctional strategies. Validity 
judgements are particular to procedures and user val|jes. But ramifications 
tf assessment characteristics bo€b types "of yalidity can be identiffed. 
,These characteristics include (1) direct measures, (2) pure measures, 
(3) culturally pli/ralltic measures, and (4) contextually, diverse measures • 
An illustrative measurement procedure is described, . ^ ^ 
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1 \, . ' ' • Varge scale assessment . 

The next several years are certain to witness Increased demand fdr ^* 
large scale assessment? of oral cownunl cation skjlls. Lajpgua^e arts edu- 
cation will be Influenced by the Inclusion of speaking and listening 
recent federally mandated'deflnitlons of "basic skills."^ .Emphasis on 
basic skills, as'popularly cpnstr'ued, is an offspV^ing of^the same sod'd-l- 
educational movement thW spawned competency based education. ' In the 
vision of this movement an important function of public schools Is, to^er- 
tify minimum levels of student achievement. In thi5- cliffiate schools and, 
teachers are held accountable for educational outcomes. primarily in {erms 
of test scores. At present there are few well established instruments 

for measuring speeth cofmiunication competencies in evaluative, nonresearch 

2 * ■ 

contexts. The manner In which the educational community develops and 

selects large scale procedures for assessing speaking and listening may ^ 

have profound consequences on the. future statu? of bral communicati or/in 

the schools. 

^Several factors are salient with respect to test construction and 

■ » 
adoption^ among them monetary cost* time* anrount of error contaminating 

» V 

measurements^ and the utility or consequences of resulting certification 

decisions. This paper addresses itself to one such factor > considerations 

] 

affecting the validity of oral comnuni cation assessments. 
J From a psychometric perspective, validity pertains to what can be 
legitimately inferred on the basis of ^^iven assessment procedure. Jn 
general^ thj?^e interdependent types of psychometric validity are recognized. 
Content validity is determined by inquiring whethei^ test itifms (including 
ellcitation procedures and scoring criteria) represent the doma4n of know- 
ledge or sjcill of interest. Criterion referenced _yalidity is demonstrated 
empirically if an ins.trument discriminates among individuals io a manner 



2 large scale assessment 

similar to results obtained 'from other related measlirements (concurrent 
^ validity) or If the instrument forecast's test-takers attributes, espec1ajlj< 
behavioral patterns, at some time after test administration (predictive 
validity)-^ infST'ences derived from an assessment procedure have construct 
validity if the test's rationale and operation are consistent with the 
.net^j^ of conceptual relations of whricH the target knowle^lge or skill Is 
^1n theory a part. Conducted by means of on-going empirical and logical 
analysis, construct validation directly queries, is this a test of what 
it purports? • 

^The relevance of a testing procedure can be examined from a pedagogical, 

as. well as psychometric point of view. In the present educational meli^u 

t^t results may or may not have Impact as feedback-utility for diagnoses 

and remediation of instructional weaknesses. But testing most surely has 

feedforward impact; classroom practkes are altered to confom to lest 

Specifications. Simple power relations in educational societies demand 

that teacjiers "teach to the test/* For better or for worse, instruction 

expands or contracts in accordance with available technologies of educational 

measurement. , ' 

* * 

It is unlikely' for example that the vigorous support presently enjoyed 
by reading programs would be possible in the absence of advanced reading ' 
A assessment methodologies. The adoption of sentence combining training in 
many quarters is motivated largely by the fact that its outcome, syntactic 
complexii^, 1^ easily'assessed. Tf\1s situation obtains despite research 
which disco/rflms '/iny direct relationship between syntactic complexity . ^ • 
and^udg.ed quality of written composition.^ Yet another instructive illu- 
stration of the effects of testing on instruction is the history of the 
Educational Testing Serv\1ce's College Board English Composition Test. ETS 
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deletecj ortginal student writihg samples from its test of writing abilit/ 

when test* constructors developed^ muUiple'choice examination which was 

rellablfe, cost efficient, a,nd highly predlctivje of writing sample ratings. 

I ^ ... 

Ht>wevdr English teachers protested that the multiple choice testing was . 

causing a reduction fn student, ccJmpOs'itf on and increased classroom emp/iasis 
on items of usage,* punctuation, and the^like included in the test. Ulti- 
mately this pressure resulted in the reinstitution of the writing sample.^ 
It is warrantable to conclude that the amount and type of^ public school 
instruction jn speech communication is currently, and will continue to be, 
^ependervt. upon the ^vailabifity and char^&cter of large scale assessment 
^procedures. . , ^ ' ^ i - 

Pedagogical validity, then, pertains to what students would acquire 
as a result of instruction designed to optimize performance on a given ^ 
assessment procedure. Two interdependent types of pedagogical validity 
may be distinguished. Curricular validity refers to the judged appropri- 
ateness of course content that is consistent with test demands. For 

example, if reconstructing -a particular communication mo^l is deemed 

% 

tangential to effecti^te speech, them a procedure which assesses ability, 
to reconstruct that moifel has low curricular validity for instruction 
intended to promote .effective speech. Instructional practices validity 
is a function of the judged appropriateness of teaching strategies used 
to implement course -objectives. If it were determined, for example, that 
group problem solving is a functional teaching strategy for promoting ef- 
fective speech, tlien an assessment'which requires experience in group 
problem so|vi<ig would h^ve high instructional practices validity in this 
s1tuatJon';\ . • - . 

Test validation, whether in' terms of pedagogical or psychometric 
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factors. Is particularistic. VnHdlty judgements hOl^ onXy for specified 



\ 
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populations, purposeSr and procedures frort which inference^ are drawn. 
Moreover, conceptualizations of communicative competenpe ar* divergent.^ 

It is therefore difficult to drsyi conclusions about factors Contributing 

% . ' • \ 

to validity in oral assessments which -are applicable over a rhnge Of testing 

needs and situations. These conclusions are best drawn by testing agencies' 

individual constituencies. Nevertheless* certain broad issues \bearing 

upon detenninations of psychometric and pedagogical validity in\oral pro- 

, ficiency testirtg can be Identified. ■ • 1 ' ' \ ^ 

Direct Measures ' \ 

Oral communication aya skill is a behavioral construct. Knowing atjout 

'speech communication is not equivalent to b^ing able to execute cormuni cation 
acts, though intellectual ap'prehensfon of pertinent variables presAnably 

r ^ ' 

itructla paper-and-pencil instrument 



enhances -skill. Mead attempted to consi 



which asked itesteres to choose appropriate communication strategies dr tq 

characterize likely outcomes in supplied situWions, l^te product wak of 

g I 1 
limited value as a standardized test. 'S.trat^fegy choice Is so hlghl^lcon- 

text-Jependent tfiat it is urvlikely 'tliat a supdlied, hypothetical sitAtton 

could provide rich enough information for choice among al terrtjatives. IThus, 

. Howie-Day found developmental trends In rationales used to justify chdices 

among alternative strategies, but not developm^tal patterning in^the 

q 

' choices, themselves. 

* It 

In a similarly Indirect a-pproach^* cognltlv^ and attltudlna'l pre^rdqui- 

V ' ' 1 1 

sites to communication can be Indexed,' but measures of underlying tra1« 

* 1 1 

* II 

may account for qnly small portions of the varlake In communication pe 

f I ■ J. 

forma/ice. A case In point Is the use of perspeciive-taking measures suqh 

as the Feffer Role Taking Task which McCaleb and Korm^in advocate as on 
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' 1 0 

* assessment tool. ^However neither correlatisfial nor Intervention studies 

^ • / . 

confirmed a strong dependency between this type of social cognitive measure 

11 ' ' 

and referential communication accuracy* 

* 

It is altogether possible, however, that an indirect assessment 
oral comftiunication skills of adequate psychometric validity could be con- ^ 
structed.„ Still, matters of pedagogical validity interpose, Adopting 

such an instrument might undermine experiential learning and the acqurji- 

^ • ■ • • 

tion of performance skills. Educators who value such practices and outcomes 

might be justified in rejecting indirect measures on this basis aTone, 

Indeed, the Speech Communicatijon Association "Criteria for Evaluating 

Instruments *and Procedures That Assess Speaking and Listening Skills" do 

demand evaluations which require students to demonstrate communication 

12 ' * 

behaviors. Among the disadvantages of performance measures of speech 

f communication are cost and measurement error, although previous experience * 

A 1^3 

in writing evaluation suggests that these difficulties can be managed. 
Prominent attacks cin the validity of such performance measures arise be- 
cause of inherent difficul t'ies in controlling extraneous sources of variaruce 
in performances, ^ ' ' ' 

If speech communication skills are conceived as including invention 
of content material, then bias^es in favor of certain subject matti^will 
be troublesome. For ex^lef^, if students are asked to speak about "My 
Summer ^Vacation," then the speaker who has worked at an archeologicaT dig 
in Greece will be at an advantage over one who Just ^'hung around and played 
a little ball." This advantage is a source of psychorpetric invalidity, 
intruding variance irrelevant to any performance skill. 

One approach to eliminating content differences ^as sources of variation^ 
is to consti*ain definitions of oral competencies to include only delivery' 
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skills and thus excluding. Invention* organization* and stylistic choice. 
Under such constrained definitions it Is proper to employ oral reading 
or memorized renditions of controlled texts as evaluation tool^. . The con- 



sequence of this testing procedure is to jeopardize pedagogical vaJidlt^ 
by^ encouraging reversion to narrowly elocutionary instruction. A secohd*/y 
approach to at least minimizing effects of content differences Is to simply 
Instruct raters to*a*tend more to effectiveness of presentation thar\ to 
content. Only experimental study* can reveal, if such instruction is an 

effective technique. A third means of reducing variatixjn due to respons,^ 

• * I 

topic entails constructing communication tasks in which all or most nec- 
essary information is supplied. Referential coinmunication accuracy exercises 
in which speakers encode supplied stimulus materials'are of this type. A 
persuasive^ task that builds in necessary inf9rma'ti;on, is presented in a 
later section of this paper. A final technique for minimizing subject 
matter biai is to ecnpl'Sy test -items which tan be presumed to be within the 
ken^ of testees. ' A question Hke* "Tell me about your favorite teacher * 
here at school" ought to' provide a more equivalent opportunity for dis- 
criminating on the basis of conmunicatton skills than an item which instead 
emphasizes differences in background Knowledge and experience lijce, *'What 
do you consider to be the most important event. in profession?] athletics 
this past year?" ' ^ 

Assessments which sample naturalistic interaction would be accorded 
high psychometric va'l^idity. In naturalistic observation qf performance, 
howeVer^ one again likely sacrifices the control and cjjnsistency needed " 
for reliable Judgements • Therefore many direct measure^ of communicdtion 
will ^nVolve contrived or hypothetical contexts. Some research instruments 
ask subjects to report about their performance as in the: instructions, 

5 
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"What would you say to this person In order to get him to return your 

14 ' * 
ball." Regardless of their value as techniques of Inquiry, such reports . 

ire not to 5e confused with techniques of performance evaluation- 'In con^. 

trast, other procedures aSk subjects to rple-play or perform as if they 

were In t|?e ipeclfled interaction situation. Role-playing tasks can re-^ 

fleet natural is tic performance skills to the extent thfet they (l)'supply*, 

sufficiently rich context ^including audience, purpose, and setting and 

C ■ 

. (2) portray. this context in a compelling enough manner to provide at least 
a. pretext for-ignoring the evaluative motivation for the exercise. A role- 
play task in which an examiner states, "Pretend I^^'m your friend and you 
want me to go to the' movies with you,!' \iolates both these conditions. 
However a. siijulated employment^ interview for a job at a fast-foo^fi^stau- 

ranti/night provide a useful speech sample. An advantage of rple-playing 

» 

a^sessm^ents is that for those who consider role-playing to be a useful 

- instructional practice* such procedures have high pedagogical validity. 

Gene' Sanford identifies a number of factors that can In^t^rfere with Instruc- 

*tional role-playing and which may also have a bearing on role-playing for 

' evaluative fJurpos^^es.^^ . . ^ 

"Pure" Measures ' , ^ 

Effective oral communication requires th6 intersection of verbal/ 

social, and logical abilities. It Is at the same time a motor 'and percfep- 

tual skill and Is also Influenced by attitudes. To search for a pure measure 

of communication competence'^s akin to the alchemists' attempts to Isolate 

elemental fire. Nevertheless*, and without expounding yet another definition 

• * 

of speech ccwmiunlca'tloij, speaking and listening are uniquely characterized 
by the spontaneous confluence of these subskills'lri the processes of con- 
veying ^nd extracting meanings. Measuring these processes constitutes an 

'JO • ' 
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especially elusive enterprise an4 consequently proposed assessmervt pro- 

cedures may capture one br another of the more accessibfe components»'of , 

conmunlcation skill employing the channel of speech only incidentally. 

One critique of currently available standardized listening tests is 

that, they are little more than reading 'tests presented orallyJ^ They 

fail to encbmpass -the conmunicative' nature of the listening process, stress- 

jng literal comprehensToh and ignoring such components of listj^ning as 

utilizing paralfinguis tic .cues, judging spe.aker attitudes^, and .forecasting 

content* Published tests of .listening may .in fact be measures of general 
* * • 

verbal ability* ^ 

Similar criticisms may be leveled at any set of evaluation 'criterja 

* that credit particular Linguistic or stylistic features In an absolute(- 

fashion. Style is cantext dependent and the mark of conmunicatl ve compe- 

" tence may indeed'be .flexibility, and not Consistency, in the use^of par- 
ly 

ticular linguistic resources. ' 'An tllOstration of this po.int is use of, 

>» 

syntactic complexity as an index of oral proficiency as proposed by^oth 

18 ^ 

Lobaa and McCaleb, Ttpugh syntactic complexity is easily measured and ' 

is a usefal indicator for many purposes, rec-ent findings argue against 
^ - ' IQ 

simplistic age ntfrm interpretations.'^' Jenson similarly founi.that speecfT 
context exerted greater impact on spoken syntactic complexit;y than did 
ability level. ' ' . 

Use of Standard English proni^ncjations, lexicons, and grammars as 
a measure o^oral conmunicatlon competence is a related, albeit more con- 
troversial, subject. . The rating instrunent in use in the Gary, Indiana 

,school district is an instance of an evaluation technique stressing Standard 

21 ~ ^ " * • 
Engl ish, dialects In an absolute fashion. However not all occasions re- 

*" * " * . 

•quire Standard English. In some contexts, speakers may be penalized for 
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speaking It. Intelligibility is a rhetorical criterion of effectiveness. 
"Cprrectness". and gentility are prescr1t)t1ve vehicles used primarily for - 
social stratification,- The two types of criteria are not necessarily 
equivalent. On the otlr^r hand, competent speakers do employ a formal style 
when a social situation is construed as of great interpersonal distance, 
Fannality may be signalled t)y a number of features of which standarjl dia- 
lect is one, but which also include distinct enunciation, erect posture, 
and use of deferential terms. An assessment (Jf oral conmuni cation skills 
need" not/be a tesl of spoken Standard English any .more than it need be 
a test of genertil verbal ability or of syntactic complexity. 

As evaluation procedures ernphasizing particular sets of l^iguistiQ 
features raise^issues of psychometric validity, so do they affect pedagog- 
ical validity., A testing emphasis on Standard English dialects, for eoc- 
ample, could result in ^ classroom concentration on "re(t\ediating" nonstand- 
ard dialects. Emphasis on syntactic complexity could result in arhetorVcal 
sentence combining drill. Depending on a community's* instructional ob- 
jectives, these may be entirely defensible goals. But if educators wisf> 
to promote in students varied repertoires of communicative resources and 
wish t().pijovide practice in contextual ly ai^propriate selection frorrj those 
repertories, then qualitative standards for language cannot beTbSolute-^^ 
^ . Personality factors also merit discussion as variables potentially 

affecting the purity of ordl Communication measures^ Tfaits like gener^^ 

■* 

alized communication apprehension and' a disposition to tentativeness may 
result in negatively evaluated speech styles. Rigid personalities are 
likely to perform poorly on certain listening tasks. If the effects of 
theke personality traits are conceptualized as extraneous contaminants of 
scores, then they ceyi be Independently measured and statistically partial led 
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out of proficiency indicators. The opposing point of view, that such per- 
sonality factors are inherent to Qornmunicatiorf skUl , can lead to far- 
reaching ranjifications concerning the schopls^ responsibility for person-, 
ality modification therapy. Yet a third portion might hoTd t||At» while 
it is beyond the scope of^public edycatjonw engagie in deliberate mass 

therapy, it is proper to offer classroom instruction which imparts to stu- 

/ • ' ' . • . . ■ • . . . 

^dents strategies' ^or coping with dysfuhctiona^ personality traits in given 
sitljatiorts.^^- " * ' ' 

Culturally Pluralistic HefSures ' . ' 

• • • 

, In Ortfl language arts, as in no other subject area, /educators are 

fprced to confrortt cultural biases of'the public schools. For it is 

' • * • ' • 

ttfrough spoken language, more than any other behavior, that individuals 

.project cultural identity <^d concommittant social status. For at Teast 

the past two decjdes educational polidy has be"^ subject to Sisyphean 

torture by the opposing forces of pluralistic egal itarfanism and demands 

for. quality control wherein the qualities defined as. minimally acceptable 

. . . ^ . . 

tend to be middle class/ technocratic. Schools have always 'operated 

to engineer the social ordei*and it is not distorting to view the *^ 

contemporary back to basics movement in this -light as well.^^ 

The equation of Standard English dialect use with commanication 
- • . , 

com|5etencA»demonstrates a particular cultural orientation, sjnce some ethni 
regional, and socio-econo;nic groups do not natively speak standard dialects 

t • 

(although all .speakers display some degree of code-switchiir^in accor.dance 
with parameters of cormiunicative content). Otfier obvious squrces of cul- 
-tural bias in oral assessment' incltjde questions which call upon a range 
of experience lirpited to.mefjjbers of paYticular subcirl tures. Mead, by 
way illustration, ^determined that minority groi^ youngsters' identified 

- •■ •■ ■ ■ * 
* ■■ V ■ - ( ■ ) • • 
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the st«^tement, "In order to get a good job you have to finish school,'* 

*" • 27 * 

/ /as a statement of 'fact more often ^an did Anglo students, 

" r^'^ LeW detectable, but of great^sr potential imp^act, are sources of 

. cultural bias atre fto differing cyltural norm? for what Is to count as 

an appropriate response or what Is corjstrued at all as a co^nunkatlve 

♦ ^ J, 

context. In certain blue-collar cultures onlV circumscribed types of 

self-disclosures are deemed appropriate, and those only jjnder highly 

'28 t ' f 

delineated circumstances. Members, of other cultures, on the other 

hand, may be proniiscuous self-disclosers. When questioned by an authority 

figure Latino youngsters may avert their eyes as a sign bf respect, 

while Anglo children may engage in eye-cdntact to demonstrate, sincerety* 

Research suggests at least some sopial cla^s diffg?enc^s in norms of 

^ verbal restraint as "opposed to s;porttaneous elaboration of responses. 

Indeed, middle clas^Nchildren ire trained in their home eQvironments 

to perform fn examination situations. Parent? typ|cally asJk'quasi-^ ^ 

questions, questions which are clearly not requests for enlight-^ent 

since-answers are paJfentlj^ known by the inquisitors^ Or children are 

given*practtce. i'ljffiMBnnO extended discourse primarily for the^ratiff- 
*. ^ -' * • 

• catiorj of their elders, ' In sum, these children learn that speakihg in ' \ 
,^ * « ^ , - " 

. the role of examinee is normal and rewarding. 

\ • ^ ' . . . ■ 

^ This pprtrayal contrasts sharply with Philips' ethnpgraphy of 

' ' • 30 ' ^ 

communication in a Native Am'erican Indian community. ^ In this minority 

culture youngs^ters typically l^arn domestic skills by silently observing 
their elders, and practicipg tn private. Demons^t rating a skill for pur- 
poses of formative evaluation is not mjrmaK Rather, a child w^ll per- 
form the new task as a matter oT course in its functional context aft^r 
m or she .is satisfied^ of InMiStery, Moreover, from early childhood 

•' ' •" '. . -) ■ 
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children are lijiTiersed in a pee!^ culture with little basis f^r the^role 
of examinee. Leadership is a shared and fluid role and it is unusual 
for a young member of thi 5 community to hold forth 'in speech for An 
extended period. For these Reasons it seems inappropriate to engai^ 
in classroom recitation. Likewise, !fi1s culture provides no foundatloTi - : 
for responding in a situation deliberately/contrived as an examination 
of oral competence. - ' 

It is difficult to avoid the conclusion that no one oral assessment 

^ ' . ^ f 

procedure can treat members of diverse cultures equlvalently. Recognizing 

and choosing to accept selected cuUDral inequities inherent in cofnnunl- 
catibn testing nfSy be a defensible posture. Reca^l1ng that validity 
* judgements pertain to the uses'Qf tests rather than to the -tests themselves, 
interpreters^are^ compel] ed to draw differing inferences from scores ob- 
tained from differing populations, however. Renderiiia^^lnter 
selecting assessment materials, criteria, and certiffcation cut-off poin.ts 
are most sensibly conducted at as local a level as feasib^le and /in 
accordance ^ith communit/ values.. 
Contextually Diverse Measures 

"The terms ''basit skills" and ."minimal competencies" connote abilities 
directly' applicable to economic, political, biological, and perhaps 
familial .survival* They tlo not translate readily to concern for self- 
concept, fulfillment, creat1vtty,''and psychic well-being. Consequently 
many competencejf based oral communication objectives refer to "life ri^es" 

like giving and comprehending travel directions^ interviewing for^eniploy- 

31 

m^nt, and unders.tartding televisionCcomrjerdals. Although not dlcJhoto- 
iDously opposed to life role objectives, speech coomunication instruction 
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has traditionally also encompassed humanistic and humanizing goals 
associated wtth the ^road liberal, arts o.f Inquiry, self-knowledge, and 
autotelic participation in conmunlty. Oaly ei.modest minority of indiv^id- 

tial5 bas frequent, Occasion to d.eliver prepared orations once embarked 

, ' • • . ■ 

upcii their ."Iffe roles." jit is not gnlikely ^hat a narrowly utilitarian 

testing domain could jleci mate attention to public oratory i^n crowded 

c\jrr1cula. *By the 5ame tokeat^^assessment procedures which elicit 

extended nonspontapeous speech jor which rely^on criteria of voice, diction 

and format J anguage style ct)uld as easily leaji to excision of iWructlon 

In dyadip and small grouf> Interaction skills. To the extent that re- ^ 

duCtion In th^ scopg of oral communication curricula Is undesirable, 

tests which sample ^rom a limited range of contmjnication "domains Incur 

^ ^ * ^' * * - 

•poor pedagogica^l validity, • v ' ' . 

Employing a. battery of tasks representing a variety of COTmunlcatlon 

contexts is a«ineaDS of prorpotlng a rich aiff varlefd speech communication 

curriculum. Increasing the\nurflber of se^l terns In this manner also 

his salubrious effects on reliability ol^ft^easurement. But this approach 

Is also very co^tl)^, ^ A more efficient sjjstem of equivalent pedagogical 

validity would be tejtee^self-selectlon, or else random assignment to 

one of a variety of communication tasks. Rigorous field testing will be 

requir^fMTrt this case to ensure that the several procedures aret in fact, 

"parallef fcgrms" and yield equivalent certification decisions. The ques- 

tlon of equlvalfency Is^one of pedagogical as well as psychometric concern*. 

For If students train to mastery on each of the tasks Ip classroom sltua- 

tlOns, the prdbab.111ty of measurement* error due to test form Is reduced. 

' 'An ^Illustrative. Measure - ^ 

The procedure described In this section Is presently undergoing Yield 
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testing In conjunction with a compe'tenoy based education demonstration 
project in^tbe State of Georgia*' It Is presented here for illustrative 
purposes only, and not as a validated Instrument proposed for adoption. ^ 

,This tesf is intended for tentb'-grade administration as an equivalent form 
option to a simulated job interview^ The task is also deliberately tied 
to pre- instruction in the Gommunication context and the evaluation criteria. 
Students engage in some guided practice in similar coomunication tasks. ■ 
to reduce expectancy effects and otherwise increase reliability, the test ' 
Is administered and scfbre^J by a pool of trained teachers from the district 
who are Assigned to test sites o-ther than their instructional assignrr^nts. 
, This task is a role-play of a public hearing before a county conmis- 

^sion. Figure 1 reproduces test materials given •to students. It was se- 
lected to sample extended persuasive discourse^in a public letting con* 
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sideved to be of life role significance. The taSk is highly structured, 
providing a degree of content consistency. It specifies purpose, audience, 
and choice of topic. While testees are apprised of sonje Informatlort of 
ev^idential vaAie, they must decide upon a position,* Invent^ arguments/and * 
are free to Qo bej^ond the information given. ^ 

In administering the procedure, the room is arrahged with a podium 
inset into the audience area and facing a row of seats occupied by student 
mock-commissioners. The "Agenda" is read aloud and stuj^nts may also ref^r 
to their printed copies. Testees speak in random order, and , the aiidlence 
is ehcouraged to applaud after each presentation, ^eo-tape apparatus 
records the perfonnances from an angle. 

Evaluation criteria appear in Figure 2, The rationale for this .scoring 
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15 • ' * large scale •assessment 



> • Insert Figure 2 about here 

system Is similar to the Primary Trait method used In the National Assess- 
' « 

^ 33 

ment of Educational -Progress' writing assessment. That Is, criteria 
are specifically keyed to the rhetorical dimensions of the cotnmunl cation 
task rather than to universally applicable communication cortipetencles. 
Since the scores are intended as diagnostic feedback to students as well 
« for certification decisions, and Irf order to enhance Inter-rater rell- 
ability, an attempt Is made to concretely describe indicators for each , 
ordere<^'lgvel of performance. Two judges Independently rate the video* 
taped presentations according each criterion equal weight. 

Many language arts educators believe^ that large scale assessment and 

* * * 

competency certification Is antithetical to effective instruction. Cer- 

tainly cofmiiinlcation does not thrive i^n distinctively evaluative ciimatA, 
Moreover, speech communication skills do not readily lend themselves to 
the types of reductlonistlc formulat16ns that ease the labor of test con- 
struction. However by ireful ly considering broad Issues *of both psycho- 
metric an^ pedagogical validity 1n concert, it may be ppisfble Jo create 
large scale assessment procedures w^ose- ultimate product 1s a heightening^ 
of the quality of -speejch cofiimunlcation in our society. 
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. Student Testing Materials 

. AGENDA,. * 

Custer 'bounty ^oard«6f Commissioners 
• * 



Cijunty ioard of^Commlssloners will hear ^Itizen^* comments about 
three pro^sa Is « Citizen^ who wish. to speak to the County Commissioners may 



The Custer 



choose only one proposal. Your ^comments should be limited to thr^e mlnvtes. 

Take a. few minuses now to thinjc about what you want to say to the Comnlft^lon. 

Prepare some notes or an outline to take with you when you 4peak* 
* 

1. Constructing a regional airport in Custer' County : 

The proposed new airport will serve mostly business and industry. It 
lill i>ring new jobs to our community. ^In o^er to build the new airport^ 
^h^ county will have to take out loans and raise taxes slightly. But 
the Federal government will pay ^half the cost* If enough flights use 
the ai;rpor^, it will pay for itself ih ^he future* The airport will 
create some noise and air pollution*^ The proposed"* site for thp newyslr** 
port is in hn agricultural area two mi le^ from the Custer County High 
School and five miles from downtown Custer* 

2« Changing the pre>^ zoning regulations: ^ 

Presently all areas o^ CiAter County arre toned intone of four ways.' "A" 

' ' ' 4/ ^ \ 

zones are for agricultoral use only* ''C' zo^es are for .commerical use 
» * y — * * 7 

^ X ' • " ' • ' 

including stores and sm^Oll businesses* "I^^ zones are the only permissible 

> / 
locations for industrial and manufacturing factories* "Rl' zones are 

residential) for private homes^^ apartment houses* This proposal 

elimin^ptes aU^ zoning regulations. With no zoning » peoI>le will be able 

to live ^oser to their work. More/areas will be Opened up for growth 

of housing) business^ and industry. Homeowners and bu^inessps will be 

/ • * 
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able to loqate wherever they choose* Property values In some areas will, 
rise. In other areas^ quiet residential neighborhoods vlll be disturbed. ' 



r;/ 



3. Establishing a C<)unty youth center 

' i * 

The proposed youth center will serie residents of Custer County^ages ten 
^ to eighteen. The center wilJ^nclude Recreational facilities such as' 
tennis and basketball courts » a swimming pool» and art studios. It vil^ 
have rooms to be used by youth clubs free of charge. The youth center 
will also house a counseling service to' help teen-agers, with emotional 
^ and health problems^ In order to pay 'for the new center, the County 
' Commission will have to r^ise property taxes by lOZ. For a home valued 
at'$30»000 property taxes will increase by $25 per year. 
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Figure 2 

PUBLIC HEARUiC FEEDBACK FORM ^ . 

STUDENT: • DATE: ' 

CLASS: ^ ^ . ■ . RATER:.; ' 

SCORE; . BfRFORMANCE STANDARD: " 

INTRQ.DUCTION: (1) none; (^) Just names proposal; (3) names proposal and attempts 
to capture Interest; (4) names proposal and provides novel approach 

PURPOSE: ^ (1) no point of view; (2) vague point of view; (3) Just- states position 
on proposal; ([4) states position with emphasis or situational qualifier 

REASONS: (1) unsupported assertion; (2) unelaborated reasons given; (3) at * 
* least one reason supported; (4) at least on redson adapted to Cocmlylon f 

perspective • • 

' . • f 

EVIDENCE (Includes common knowledge): (1) no Information or ottvloualy Incorrect. ' 
Information; (2) Information of questionable validity *or relevance; 
(3) clear and pertinent Information; (4) Information likely to be new or 
or Interest to Commission 



ORGANIZATION: (1) Ideas wholly unrelated; (2) Ideas implicitly or theoatlcally 
related; (3) logical sequence or simple transitions; (4) proper emphasis 
and explicit connections between Ideas 

OBJECTIONS: (1) does not acknowledge reservatlonsj (2) acknowledges but does 
not refute reservations; (3) refutes at least ^one reservation; (4) refutes 
most Important reservation from Commission's oilnt of view 

CONCLUSION: Ml) no conclusion or merely states that remarks are finished; 
(2) just thanks Commission; (3) restates position and offers thanks; 
(4) summarizes or concludes memorably and offer thanks ' ^ 

LANCU/^CE STYLE: (1) slang or Incomprehensible on several occasions; (2) very 
va^ue or dlstractltjg "fillers'^ (3) fluent^ appropriate formality; 
(4) vivid phrasing, highly comprehensible 

GESTURES: (1) distracting. manner Isms or posture; (5) no eye contact with e 
C^lssloners;' <3) eye contact established, comfortable posture; (. 
.(4) facial, body, or hand gestures used for empHasls^or Illustration 

ORAL EXPRESSION: (1) monotone; (2) Inappropriate or distracting inflection 
on several o<?taslons; (3) natural variation In Inflection; (4) tone 
of volce/^expxesses conviction or emphasis 

• ' • # 

SPEECH RATE AND VOLUME: (1) Inaudible; (2) rate" too Ust or too slow - 
distracting; (3) natural speech rate and Volume; (4) variation in 
• rate or^valume used for added expressiveness 
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